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Abstract 

A  computer  program  is  described  that  uses  maximum  likelihood  methods 
to  estimate  item  parameters  under  the  three -parameter  logistic  model  for 
item  characteristic  curves.  The  procedure  also  estimates  examinee  ability 
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A  computer  program  is  described  that  uses  maximum  likelihood  methods  to 
estimate  item  parameters  under  the  three-parameter  logistic  model  for  item 
characteristic  curves.  The  procedure  also  estimates  examinee  ability  levels. 
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A  COMPUTER  PROGRAM  FOR  ESTIMATING  ITEM  CHARACTERISTIC  CURVE  PARAMETERS 
USING  BIRNBAUM' S  THREE -PARAMETER  LOGISTIC  MODEL* 

Given  the  item  responses  of  a  group  of  examinees  to  a  set  of  items, 
the  computer  program,  LOGIST,  estimates  the  parameters  of  the  item 
characteristic  cui've  for  each  item,  using  Birnbaum’ s  three-parameter 
logistic  model.  The  parameters  are  estimated  by  maximum  likelihood,  using 
an  improved  version  of  the  method  described  by  Lord  (1968).  The  program 
was  initially  written  by  Diana  Lees  and  modified  later  by  Marilyn  Wingersky. 

The  parameters  of  item  characteristic  curves  are  needed  for  effective 
design  of  tailored  tests,  for  comparing  the  relative  efficiency  of  two 
tests  or  of  two  scoring  methods  at  any  specified  ability  level,  and  also 
for  computing  a  scoring  weight  for  each  item  that  will  maximize  the  effec¬ 
tiveness  of  the  test  at  some  specified  ability  level. 

The  set  of  items  used  to  estimate  the  item  parameters  must  meet  the 
assumptions  underlying  the  Birnbaum  model.  These  assumptions  are  (Lord, 
1968) 

"1.  The  test  items  have  only  one  psychological  dimension  in  common  ... 

2.  The  test  items  are  scored  either  right  or  wrong. 

5.  The  probability  that  an  examinee  will  answer  a  given  item  cor¬ 
rectly  is  a  three -parameter  logistic  function  of  the  trait 
being  measured." 

(See  equation  1  in  Appendix  A.) 

> 

*This  research  was  sponsored  in  part  by  the  Personnel  and  Training 
Research  Programs,  Psychological  Sciences  Division,  Office  of  Naval 
Research,  under  Contract  No.  N00014-69-C-0017,  Contract  Authority 
Identification  Number,  NR  No.  150-305,  and  Educational  Testing  Service. 
Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose  of  the 
United  States  Government. 
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Method 

The  parameters  estimated  by  the  program  are  the  ability  levels,  8&  , 
for  examinee  a  (a  =  1,2, and  three  parameters  for  item  i 
(i  •=  1,2,  ...,n)  .  The  item  parameters  are  ai  ,  the  item  discriminating 
power,  b^  ,  the  item  difficulty,  and  ,  the  pseudochance -score  level. 

The  c^  may  be  held  constant  and  only  the  a^  and  b^  estimated.  Al¬ 
though  the  origin  and  unit  of  measurement  for  the  Q's  are  arbitrary,  leaving 
only  N  -  2  unique  Q’s,  all  N  Q's  are  estimated.  The  estimated  Q's 
are  linearly  transformed  to  prevent  uncontrolled  drift. 

The  likelihood  equations  for  these  parameters  are  solved  iteratively 
in  stages  using  a  modified  scoring  method  (Rao,  1965).  Each  stage  consists 
of  two  parts;  the  first  part  solves  for  new  0  estimates  holding  the  item 
parameters  constant,  the  second  part  uses  the  new  9  estimates  and,  holding 
them  constant,  computes  new  estimates  of  the  item  parameters.  The  initial 
starting  values  for  the  procedure  are  computed  by  the  program.  The  likeli¬ 
hood  equations  and  detail. s  about  the  procedure  are  given  in  Appendix  A. 

Convergence 

It  is  important  to  have  a  sizable  number  of  items,  perhaps  n  >  40  , 
and  a  large  number  of  examinees,  perhaps  N  >  1000  .  Unless  the  sample  is 
large,  an  estimated  item  discriminating  power  may  increase  without  limit, 
in  which  case,  it  may  be  necessary  to  remove  the  unstable  item  before  the 
process  will  converge. 

To  imorove  convergence,  the  program  automatically  excludes  examinees 
with  perfect  number-right  scores  on  those  items  included  in  the  estimation 
procedure.  The  maximum  change  of  any  parameter  in  an  iteration  is 
restricted  by  the  program  to  dampen  wild  fluctuations. 
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The  program  does  not  check  for  oscillating  or  diverging  parameter  estimates 
and  it  is  therefore  advisable  to  periodically  check  the  progress  of  the 
iterations.  A  restart  feature  is  built  into  the  program. 

The  procedure  is  considered  converged  when  the  changes  in  the  param¬ 
eters  become  less  than  the  truncation  errors,  causing  the  likelihood  func¬ 
tion  to  fluctuate.  The  process  ends  when  the  likelihood  decreases  in  at 
least  two  out  of  four  successive  stages.  This  may  occur  in  20  or  30 
stages.  This  convergence  criterion  is  a  strict  one--fewer  stages  yield 
quite  satisfactory  results. 

Program  Considerations 

The  computer  program  is  written  in  FORTRAN  IV  with  several  IBM  360  assembly 
language  subroutines.  The  assembly  language  subroutines  are  for  bit  manipulation, 
timer  control,  and  non-FORTRAN  input.  The  program  requires  150K  for  compilation 
in  FORTRAN  G  and  about  126k  core  for  execution  on  an  IBM  360/65. 

The  dimensions  of  the  program  allow  for  3000  examinees  and  96  items.  The 
item  responses  are  stored  in  memory  in  binary,  32  responses  per  machine  word. 

The  program  isn't  fast  but  it  is  feasible.  Some  idea  of  the  execution  time 
on  a  360/65  can  be  gotten  from  the  following  information.  It  took  approximately 
150  seconds  per  stage  to  run  64  items  with  2861  examinees,  requiring  about  20 
stages  to  converge.  Another  set  of  data  with  42  items  and  2858  examinees  averaged 
90  seconds  per  stage. 

A  test  is  made  at  the  end  of  each  stage  to  see  if  there  is  enough  time  for 
another  stage.  If  not,  the  current  values  of  the  parameters  are  punched  and  the 


program 


exits . 
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Input* 

The  only  input  necessary  to  start  the  procedure  is  the  item  responses  for 
each  examinee  scored  1  if  right,  0  otherwise.  Omits  are  treated  as  wrong. 

The  input  data  file  may  be  in  binary  or  BCD.  Input  options  include  selecting 
items  from  those  read  and  omitting  examinees  from  the  input  data  file. 

To  restart  the  procedure  where  the  previous  run  ended,  the  punched  output 
from  that  run  is  necessary.  This  output  may  be  modified  by  removing,  adding, 
or  changing  items  or  changing  thetas. 

Output* 

The  printout  contains  sufficient  information  for  following  the  progress 
of  the  procedure.  Some  of  the  information  printed  for  each  stage  is:  a)  all 
6  '  s  whose  absolute  value  is  greater  than  3;  b)  the  value  of  the  likelihood 
function  after  the  theta  estimation,  and  again  after  the  item  parameter  estima¬ 
tion;  c)  the  maximum  value  of  the  derivative  of  the  likelihood  function  for 
each  type  of  parameter.  During  each  item  parameter  iteration,  the  previous 
value,  the  current  value,  the  change,  and  the  value  of  the  derivative  of  the 
likelihood  function  are  printed  for  each  parameter.  At  the  end  of  each  run  the 
final  item  parameters  and  the  final  thetas  are  both  printed  and  punched. 

Availability 

A  copy  of  the  program  may  be  obtained  upon  written  request  from  the 
authors.  Educational  Testing  Service,  Princeton,  New  Jersey  08540.  The 
user  must  provide  a  tape  on  which  the  program  will  be  loaded  in  80 -character 
card  images.  The  user  must  specify  whether  the  tape  should  be  blocked,  in 
EBCDIC  or  BCD,  7-track  or  9-track,  and  the  tape  density  and  parity. 


*A  detailed  description  is  given  in  Appendix  B. 
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Appendix  A.  Formulas  and  Computational  Details 


The  three -parameter  logistic  model  defines  the  probability  that  examinee 
a  will  get  item  i  right  as 

(1)  Pia  =  Prob(xia=  l|ai,biJci;©a)  W  ^  ♦  (l  - 

where  $  is  the  logistic  function 


(2>  ?(,U)  ■  mknrj 

and  (3)  (i&  =  Da^  -  • 

D  is  a  constant  approximately  equal  to  1.7,  ©a  is  the  ability  level 
for  examinee  a  ,  a^  is  the  item  discriminating  power,  b^  is  the  item 
difficulty,  and  c^  is  the  pseudo-chance  score  level  for  item  i  .  The 
response  of  examinee  a  to  item  i  is  x^a  coded  1  if  the  response  was 
correct,  0  otherwise. 

Given  the  N  x  n  matrix  of  item  responses  xia  ,  the  program  computes 
estimates  of  a  ,  b  ,  c,  ,  (i  =  1,2,  ..  .,n)  and  9  (a  =  1,2, ...,N) 
that  maximize  the  likelihood  function 


L  = 


n  N 
II  n 
i=l  a=l 


1-x 

Jia 


ia 


> 


by  maximizing  the  log  of  the  likelihood  function, 

iog  L  =  £  Z  (x.a  log  Pia  +  (1  -  xia)  log  Qia)  , 

1=1  a=l 

where  N  is  the  number  of  people,  n  is  the  number  of  items,  and  is 

1  -  Pia  .  There  are  N  ability  parameters  and  3n  item  parameters  to  be 


estimated. 
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Method  of  Estimation 

A  variation  of  the  scoring  methoa  described  in  Rao  (19^5)  i*  used  to 
compute  the  parameter  estimates.  With  trial  values  for  the  parameters, 
the  scoring  method  computes  additive  corrections  to  the  values  by  solving 
a  set  of  simultaneous  equations.  This  process  is  repeated, correcting  the 
corrected  values  until  the  size  of  the  corrections  becomes  negligible.  The 
estimated  values  will  be  denoted  by  writing  the  parameters 'with  "hats"  over 
them.  The  matrix  of  coefficients  for  the  linear  equations  is  the  information 
matrix.  A  direct  application  of  the  scoring  procedure  to  this  problem 
would  require  solving  an  N  +  3n  system  of  linear  equations. 

To  simplify  the  problem,  the  linear  equations  are  solved  in  two 
sections.  In  the  first  section  the  item  parameters  are  held  constant  and 

A 

the  equations  solved  for  the  corrections  to  the  ©'s  until  the  corrections 

A 

become  negligible.  In  the  second  section  the  ®'s  are  held  constant  and 
the  item  parameters  corrected  until  the  corrections  become  negligible.  The 
two  sections  combined  we  called  a  stage.  The  stages  are  repeated  to 
adjust  for  dependencies  between  the  abilities  and  item  parameters  until  the 
likelihood  function  no  longer  increases. 

There  are  three  types  of  convergence,  the  convergence  of  the  corrections 
to  the  abilities  within  a  stage,  the  convergence  of  the  corrections  to 
the  item  parameters  within  a  stage,  and  the  convergence  over  stages  of 
the  likelihood  function  to  a  maximum. 

/S 

Explicitly  a  stage  consists  of  four  parts:  computing  new  9's,  scaling 

A  A  A 

the  9’s,  a’s,  and  b's,  computing  new  item  parameter  estimates,  and  testing 
for  convergence  of  the  likelihood  function.  The  following  sections  explain 
the  four  parts  in  detail.  To  simplify  writing,  the  estimated  parameters 
will  be  written  without  "hats"  for  the  rest  of  the  paper. 


Estimating  the  0*3 
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Since  the  O' s  are  Independent  of  each  other  they  can  be  estimated  one 
at  a  time.  The  increment  for  9  ,  ,  is  found  by  solving  the  following 

equation. 

(5)  aea  •  i(ea)  *  s(oa) 


where 


(6) 


s<®.> 


d  log  L 

~5o 

a 


=  D 


n 

Z 

i-1 


fc 


la 


~  cl)(xia 

a  -  ci)p; 


Pia> 


la 


and 


(7) 


i(ea)  =  c(s(oa)2) 


^Pia  '  cj)  Sla 
(i  -  = 


r  r- 1 

Let  0  denote  the  new  value  of  9  and  9  the  previous  value 

6L  & 

t  r— 1 

of  9  ,  then  9=9  +  £8  .  The  iterations  for  an  examinee  continue 

a  a  a  a 

until  either  the  maximum  number  of  iterations  is  reached  or  !^®al  becomes 
less  than  some  tolerence  specified  by  the  user. 

Restrictions  on  9  Estimation 


9  . 

a 


The  following  restrictions  are  placed  on  the  magnitude  and  change  in 


•A**  • 

(a)  If  9*  tmn  a  dll rr*»r**nt.  «ign  from  9^“'  aiu!  1  >  '‘I'01'  MP'* 

^  ®r‘  ‘  1  #  Bi«"  <®m’3  )  * 

(b)  The  Amount,  that  9^  may  chAnge  i«  rp*trict.*>d  «uch  that. 

|A9|  <  |.|  *  ©‘'"l|  »  P  .  tf  Ni  Is  grpAt,«r  than  thin  Amount, 

“  ft  R 

then  Mb  9*'  -  ©I’1  '  (1*1  *  9*"'1'|  t  R)  *  »lgn(ftf)  . 

(c)  The  range  of  9*  is  restricted  no  that  TIjO  -  9**  <  TH1  ,  where 

ThO  -  -10 10  and  Till  *  .10 10  unless  different  extrema  have  been 
read,  if  9^  >  Till  ,  9*'  la  sat  to  THI  .  If  #[  <  l'UO  ,  9[ 

la  set  to  TLO  . 

A  massage  is  printed  if  any  adjustments  are  made  in  the  0’s. 

Scaling 

Since  the  origin  and  unit  of  measurement  of  the  O's  are  arbitrary, 
there  are  only  N  -  2  unique  0*a.  However,  all  N  O's  are  estimated  and 
then  linearly  transformed  to  prevent  uncontrollable  drift.  The  a's  and 
b's  are  adjusted  accordingly. 

The  standard  method  of  normalizing  using  the  arithmetic  mean  and 
standard  deviation  isn't  used  since  a  few  of  the  9's  may  be  increasing  or 
decreasing  without  limit.  Instead  the  0^  are  scaled  so  that  they  are 
centered  around  the  midpoint  of  those  0  lying  in  the  Interval  [-2/3,2/31  ; 
and  so  that  for  2/3  of  the  0  ,  |0  |  <  1  .  Thi3  is  accomplished  approxi- 
mately  using  the  following  algorithm. 

l)  Let  P23H  be  the  proportion  of  the  0^  which  satisfy  the  condition 
that  -  2/3  <  6  <0  and  let  P23P  be  the  proportion  of  the  0  which 
satisfy  the  condition  that  0  <  9fi  <  2/3  •  A  midpoint,  X23,  is  determined 
(by  interpolation)  such  that  P23P  ■  P23N  -  l/2  and  all  the  6  are  shifted 
shifted  by  the  amount  X23,  0*  =  9  -  X23  . 
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;')  t*t  K*3  bp  the  proportion  of  the  9*  which  satisfies  the  condition 
that  -  p/3  <  9*  <  2/3  and  lot  PI, 13  be  the  proportion  which  aebiafiaa  the 
condition  that  1..13  <  9*  <  I.15  .  Then  a  acalr  factor,  X  ,  is  determined 
(by  interpolation)  auch  that  p/3  of  the  9*  aatiafy  (approximately)  the 

A 

condition  that  -1  <  9  <  1  .  All  the  9  are  then  acaled  by  X  ,  so  that 

*■  f\  —  a 

®r *  ®jA  • 

The  item  parameters  are  then  acaled  appropriately,  i.e.,  a^  »  a^X 

b  -  m 

b^  ■  -*y*  —  «  Aa  the  process  reaches  convergence  the  scaling  stabllir.es 
so  that  P2JP  -  1/2  ,  P23N  -  l/2  ,  P23  -  l/2  ,  P115  -  3/4  >  X2J  -  0  , 
and  X  -  1  . 

Item  Parameter  Estimation 

Using  the  new  scaled  0’s  and  holding  them  constant,  new  item  param¬ 
eters  are  estimated.  Since  the  items  are  independent,  each  item  can  be 
estimated  separately.  The  new  item  parameter  estimates  are  found  by 
solving  the  following  3x3  system  of  linear  equations.  This  is 
iterated  until  the  maximum  number  of  iterations  has  been  reached,  has 

become  greater  than  4  or  [Aa^l  <  .01  . 

The  equations  to  be  solved  are: 

^aiI(ai>ai)  +  ^iI(aiJbi)  +  ^il(ai>ci)  =  S(ai) 

(8)  AaiX(a1,bi)  +  Ab^b^b.)  +  A^Kb^^)  =  S(bi) 

Aail(ai,ci)  +  Abil(bi,ci)  +  Acil(ci,c1)  =  S(ci) 


where 


(9) 


,  ,  i>  loa  L 

Q(*i>  * 


N  %  "  ci>  ,  D  \ 

y  fo  -  b.)  — **s — —  (xu  -  Pi»  ’ 


t-™  b  (©  -  bt)  - 

1  '  °i  a-1  *  1 


s(bi) 


d  log  L 
db^ 


1  -  c 


>i  r  (F1»  '  °jl 


i  #.■! 


S^)  - 


b  log  L 


1  -  c 


*  *i.  •  ?i. 

-  E  p 

i  a=l  la 


2 


_2  N 


»2  la 


.  V  -  id 

^7  a=l  <#*  '  ^  '  0i’  ^ 


I(ai,bi)  -  C(S(ai)S(bi)) 


D2a.  N  .  .  v2  S-a 

7 - ^2  S.  <9a  -  bi)(Pia  -  Ci) 

(1  -  c i )  a=x 


iCa^c^  =  e(S(a.)S(Ci)) 


- 5 — -  £  (®a  -  V<Pla  -  ci> 

(1  -  ci)2  a=l  ia 


(15) 


-A8- 


1)  Aa  <  |.l  •  a^"1!  +  .2 

2)  Abi  <  |.2  •  +  .4 

3)  Ac.^  <  CLAMDA  •  CLAMDA  is  read  in.  A  good  value  seems  to  be  .06. 
Restrictions  on  range  for  item  parameters: 

1)  For  a^  ,  the  maximum  value  AHI  =  10 10  and  the  minimum  value 

x* 

ALO  =  0  are  set  by  the  program  unless  other  values  are  read.  If 
exceeds  the  maximum  and  the  previous  value  was  at  the  maximum,  the 
simultaneous  equations  for  item  i  are  resolved  for  bi  and  c  omitting 
the  a^  row  and  column.  Otherwise,  if  a^  goes  out  of  range,  it  is  set 
to  the  bound  exceeded. 

2)  For  bi  ,  the  maximum  value  BHI  and  the  minimum  value  BLO  are 

set  by  the  program  with  BLO  =  -10^  and  BHI  =  +10^  ,  or  other  values 
for  the  extrema  may  be  read.  If  b^ ^  >  BHI  ,  b^* ^  is  set  to  BHI  . 

If  b^,J  <  BLO  ,  b^'1  =  BLO  . 

3)  For  ,  the  maximum  value,  CHI  ,  may  be  read  or  it  is  set  by 

the  program  to  1  divided  by  the  number  of  choices.  Two  minimum  values 

for  ^  are  read.  One,  CLOM  ,  is  used  when  bi  <  0  ,  the  other  CLOP 

I*  1 

is  used  when  >  0  .  If  becomes  less  than  the  appropriate  mini¬ 

mum,  indicating  that  it  is  poorly  determined,  it  is  set  to  CHI  ,  which 
is  presumably  a  better  estimate  than  the  appropriate  CLO  .  The  other 
parameters  for  this  item  are  not  adjusted,  and  is  held  constant 

for  the  rest  of  this  stage  and  in  the  following  stages.*  If  °  >  CHI  , 
it  is  set  to  CHI  . 

*This  procedure  is  currently  being  improved. 
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Item  Parameter  Options 

The  following  options  are  available  for  item  parameter  iterations 
(Usually  the  c's  are  held  constant  for  the  first  few  stages  and  then 
allowed  to  vary,  but  this  isn't  done  automatically  unless  the  procedure 
has  converged . ) : 

a)  All  item  parameters  may  vary. 

b)  All  c's  are  held  constant  until  the  procedure  converges,  then  the 
c's  are  allowed  to  vary. 

c)  The  a's  and  c's  are  held  constant  until  the  procedure  converges, 
then  the  a's  varied,  then  the  c's  varied. 

d)  The  c's  are  held  constant  throughout. 

e)  Items' will  be  removed  from  the  Iteration  procedure  at  the  end  of 
a  stage  under  the  following  circumstances: 

1.  If  the  matrix  of  coefficients  for  this  item  is  singular,  the 
item  will  be  removed.  If  there  is  more  than  one  such  item, 
the  last  one  encountered  is  removed.  The  others  will  be 
picked  up  in  succeeding  stages  if  the  singularity  still 
exists . 

2.  At  the  request  of  the  user  items  may  be  removed  if  becomes 
greater  than  a  specified  value. 

Initial  Estimates 

The  following  initial  trial  values  are  computed  by  the  program. 

The  initial  estimates  for  the  0  are  the  normalized  number-right  scores, 

& 

i.e«, 


N  =  L  x. 
a  i=1  ia 


N+  -  x 


a  s. 


where  x  and  s^  are,  respectively,  the  mean  and  standard  deviation 
of  the  number  right  scores.  The  N+  ,  x  ,  and  s  are  printed. 

&  X 

The  initial  estimates  for  the  item  parameters  are: 

1)  All  of  the  a's  are  set  to  1. 

2)  The  are  computed  from  the  conventional  item  difficulties 

(proportion  of  correct  answers)  using  the  following  formulas: 

=  (  Vl  +  a^)  7^  =  V2  ,  since  all  ai  =  1  initially.  The 

7^  are  computed  from  the  following  formula  using  the  approximation 
given  in  Hastings  (1955),  P*  191> 


W  Q 

„  _  1  f  -t  /2 

rti  -  2  it  J  e 


where 


it.  =  —  £  x. 

1  N  a=l  la 


3)  If  good  estimates  for  the  c^  are  available,  they  can  be  used. 

Otherwise  all  c's  are  set  to  l/K  ,  where  K  is  the  number  of  choices 
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per  item.  The  c's  can  be  set  to  0,  making  equation  1  the  two 
parameter  logistic  model. 

Since  the  initial  estimates  of  the  abilities  are  better  than  the 
initial  estimates  for  the  item  parameters,  the  procedure  should  be  started 
with  the  second  part  of  a  stage  (estimating  the  item  parameters). 

Termination  Procedures 

The  program  will  terminate  and  punch  all  parameters  for  the  following 
reasons: 

1.  The  process  has  converged  when  the  likelihood  function  has 
decreased  twice  out  of  four  successive  stages.  At  this  point 
all  of  the  derivatives  of  the  likelihood  function  should  be 
approximately  0.  Two  more  stages  are  computed  with  detailed 
printout . 

2.  The  maximum  number  of  stages  have  been  completed. 

3.  There  isn't  enough  time  for  another  stage. 

4.  The  maximum  number  of  items  have  been  removed,  if  any  were 
allowed  to  be  removed.  Items  may  be  removed  if  the  matrix 

of  coefficients  was  singular  or  if  the  ai  exceeded  a  maximum 
of  10  or  a  maximum  read  in. 


The  procedure  may  be  restarted  using  the  punched  output. 


Appendix  B 


Input 


Options 


The  user  has  some  control  over  the  execution  of  the  program*  To 


keep  track  of  the  progress  of  the  procedure,  the  user  can  specify  that  only 

a  given  number  of  stages  be  executed  and/or  that  the  program  execute  for 

only  a  specified  number  of  seconds.  The  maximum  number  of  iterations  for  an 

item  in  a  stage  must  be  specified*  The  iterations  for  an  ability  parameter 

are  terminated  by  either  exceeding  the  maximum  number  of  iterations  read  in 

or  when  ^9  is  less  than  a  user  specified  tolerance*  The  first  stage  in 
a 

a  run  may  begin  with  either  estimating  the  abilities  or  with  estimating  the 
item  parameters.  For  any  item,  ^  ,  or  ai  and  c^}  may  be  held  constant 
while  the  remaining  parameters  vary.  The  bounds  on  the  ranges  of  the  param¬ 


eters  and  the  number  of  items  that  may  be  removed  if  exceeds  a 
specified  maximum  may  be  supplied  by  the  user.  The  initial  trial  values 
may  be  computed  by  the  program  or  supplied  by  the  user. 


The  item  response  input  must  contain  one  record  or  card  for  each 
examinee  with  the  responses  coded  1  if  right,  0  otherwise.  The  item 
responses  may  be  on  tape,  disk,  or  cards  in  BCD  or,  if  on  tape  or  disk,  in  bi¬ 
nary.  Selection  of  examinees  from  the  data  file  is  possible  to  the  extent  of 
skipping  the  first  m  examinees  and  selecting  every  i  one  thereafter,  where 
m  and  i  are  specified  by  the  user.  Specific  examinees  may  also  be  omitted 
from  the  input.  The  items  to  be  selected  from  the  input  record  are 
specified. 

The  procedure  can  be  restarted  using  the  punched  output  from  a  previous 
run*  This  output  may  be  modified  after  being  read.  The  modifications 
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possible  are  changing  9  values,  removing  examinee*  and  their  9'«, 
adding  examinees  and  their  9'n,  changing  item  parameter*  and  adding  or 
removing  items. 

The  matrix  of  item  responses  may  he  printed.  However  it  will  he  in 
hexadecimal.  The  matrix  of  coefficients  to  the  linear  equations  may  he 
printed  for  as  many  stages  as  desired. 

Minimum  Setup 

The  following  setup  will  use  the  program  with  a  minimum  of  options. 

In  this  setup  the  program  computes  the  trial  values,  removes  no  examinees 
from  the  item  response  data  read  on  unit  10,  seta  the  initial  c'a  to  l 
divided  by  the  number  of  choices  per  item  and  does  not  use  a^  ns  a 
criterion  for  removing  items. 

Card 

1  Punch  card  1  as  described  in  setup. 

2  Punch  cols.  1-5,  6-10,  16-20,  35,  75,  and  80  as 
described  in  setup.  Leave  the  rest  of  the  card  blank- 

3  Punch  card  3  a3  described  in  setup.  A  good  value  for 
CLAMDA  is  .06,  for  CIOM  is  .05,  and  for  CLOP  is  0. 

4  Punch  as  described  in  the  setup* 

5  Cols.  1-15  should  be  blank.  Set  col.  20  to  the  maxi¬ 
mum  number  of  items  that  can  be  removed.  The  rest  of 
the  card  should  be  blank.  This  is  card  6  in  setup. 

6  If  col.  80  of  card  2  is  1,  punch  card  11  as  described 
in  setup . 


•'JtV 

or*  uumt i 


CANI1  U 

001$ 

FORMAT  (10A0,  PX,  It,  If) 

1-00 

TITM 

01 -OP 

(not  read) 

63-05 

n  -  number  of  items 

n  ^  90 

06 -'ft' 

N  »  number  of  examinees 

N  <  3000 

CARD  8i 

001$ 

pgfoMAT  (lfel5) 

1-5 

MAXST  *  Maximum  number  of  stages 

6-10 

MAX IT  m  Maximum  number  of 

iterations  per  item 

par  stage 

15  blink 

19-20  NCH  *  0  If  o  -  0 

■  1-9  m  Number  of  ehoioea/item 

>  10  LD4  (ool.  60)  must  be  l.  8i«  oirdi 

10  for  o  input.  NCH  must  be  >  .10 
if  the  output  from  i  previoui  run  is 
being  reid. 

25  NIT  *  0  begin  the  run  with  the  estimation  of 

the  item  parameters. 

>  0  Begin  the  ran  with  the  estimation  of 

abilities. 

26-30  N^T  -  Number  of  examinees  to  be  skipped  at 

the  beginning  of  the  tape 

35  INPUT  *  0  Responses  are  in  BCD  on  unit  10. 

-  2  Responses  are  on  cards. 

Responses  are  in  binary  on  unit  10. 

See  description  of  item  response  input. 


5 


CAM  Pi 


OOlfl 
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J6-40 


41-45 


46-50 


FORMAT  (.1615) 

m  Number  of  examinees  to  be  shipped 

when  reading  tape,  after  eliminating 
the  first  "N0T"  examinees. 

If  NSKIP  >  0  ,  then  take  every 
"N8KIP"-th  examinee,  e.g.,  if 
N8KIP  "  4  then  the  4th,  8th, 

18th,  etc.  examinees  will  be 
used.  If  NSKIP  -  1  and  Nflfa  »  0  , 
the  first  N  examinees  will  be 


used. 

JC  -  0  Vary  all  item  parameters  unless 
NCH  -  0  then  vary  only  at  and 

V 

.  1  Hold  constant  until  the  pro¬ 

cedure  converges,  then  vary  • 

„  2  Hold  a^  and  constant  until 

the  procedure  converges,  then  hold 
c  constant  until  the  procedure 
again  converges,  then  vary  all 
parameters,  unless  c^  =  0  . 

»  3  Same  as  NC  =  1  ,  but  don't  vary  c±  t 
HAST  =  Number  of  last  stage  fox'  which  the 

matrix  of  coefficients  for  the  item 
parameters  is  printed* 


Detailed  Input 


CARD  2s 


FpfoMAT  (1615) 


51-55 


Compute  initial  estimates  of  G^'s. 
Read  in  initial  estimates  of  G  _'s- 


56-60 


61-65 


66-70 


71-75 


76-80 


See  CARD  5* 

-  2  Read  in  the  output  from  a  previous  run. 

IA  ■  0  Set  ai  *  1.0  for  all  items. 

/  0  Read  in  initial  estimates  of  a^'s. 

See  CARDS  8  . 

IB  »  0  Compute  initial  estimates  of  b^'s, 

/  0  Read  in  initial  estimates  of  b^s. 

See  CARDS  9. 

IY  =0  Do  not  print  matrix  of  item  responses - 

j/  0  Print  matrix  of  item  responses. 

NOTE:  This  print  is  in  hexadecimal. 
IT0NE  =  0  Only  1  iteration  per  ©a 

>  0  Number  of  iterations  per  9 


LIM  =0 


Use  the  following  as  limits  on  the 
parameters 


1CT10  <  &i  <  1010 
10-10  <  bt  <  1010 
10 '10  <  9a  <  1010 
ci  <  K 


where  K  =  NCH  (col.  20). 

^  0  Read  the  limits.  See  CARD  11. 


Detailed  Input 
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CARD  5 

COI£ 

FORMAT  (E15.8,  F6.0,  5F6.2) 

1-15 

T0L1-  iterate  on  until 

a 

<  TOLL 

16-21 

TIMAX  =  Maximum  time  for  run  in 

seconds  minus 

20  seconds 

22-27 

CLAMDA  -  maximum  allowable  change 

in  Ci 

28-33 

CLOM  -  minimum  value  for  c. 

if 

b.  <  0 

i 

i  — 

34-39 

CLOP  -  minimum  value  for  ci 

if 

b.  >  0 

1 

CARD(S)  4: 

COLS 

FORMAT  (2613,  2X) 

(as  many  as 

4  cards) 

1-3 

ITEMNO(l)  No.  of  1st  item 

4-6 

ITEMN0(2)  No.  of  2nd  item 

ITEMNO ( LITTLN )  No.  of  nth  item 
This  card  controls  the  selection  of  items  from 
the  input  record,  and  assumes  the  items  on  the 
input  record  are  numbered  consecutively  from  1 
to  the  number  of  items  in  the  record. 

CARD(S)  5  Omit  if  IT  (col.  55  of  card  2)  does  not  equal  1. 

card  a  set  col.  2  to  0 

card  b  col.  64-65  number  of  items 

66-72  number  of  0's  being  read 

card  c  blank 

card  d  FORMAT  for  reading  0's 

card  e  6's  punched  in  format  given  in  card'd. 


card(s)6  cols 
1-5 
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F0FMAT  (1615) 

NKILL  =  number  of  examinees  to  be  removed  from 

the  item  response  data  file.  (  NKILL  <  99  •) 
6-10  KILL(l)  -  sequential  position  on  the  input  data 

H-15  KILL(2)  file  of  the  examinees  to  be  removed. 

*  If  NKILL  is  0,  then  KILL(l)  must 

be  left  blank.  The  total  number  of 
KILL  (NKILL)  examinees  to  be  read  is  N  on  card  1 

plus  NKILL. 

TffTT.T.  -  blank  field  following  the  number  of 
the  last  examinee  to  be  removed. 

MAXOUT  -  maximum  number  of  items  which  may  be 
removed 

ICKA  =  0  Do  not  check  a^  as  a  criterion  for 
removing  items 

>  0  Remove  item  i  if  >  ALIM 
LA  =0  ALIM  =  10 
/  0  ALIM  =  LA  . 

An  example  of  this  card  will  make  it  clear.  Suppose  from  a 
data  file  of  100  examinees,  examinees  47  and  99  are  to  be 
removed.  Then  N  on  card  1  is  98*  NKILL  is  2,  KILL(l)  is 
b7  and  KILL(2)  is  99-  MAXOUT  would  be  punched  in  cols.  21-25- 
CARD(S)  7  Omit  if  INPUT  (col.  55,  card  2)  is  0  or  5 
Item  responses  are  on  cards  in  BCD. 

Card  1:  Format  for  item  response  cards  in  Ak  format 
Card  2:  Item  responses  for  each  examinee  punched  in  the  format 
specified  in  card  1.  No  item  selection  is  done..  No 
examinees  can  be  removed. 
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Detailed  Input 

CARD(S)  8:  Initial  values  for  (Omit  if  IT  =  2  or  IA  »  0 

on  card  2) 

card  a  cols.  1-80  Format  for  reading  in  a’s.  The  a's  must  be 

read  as  floating-point  numbers. 

card(s)  b  a(i)  ,  (  i  -  1,  LITTLN  )  -  Punched  in  the  format 

specified  in  card  a. 

CAKD(S)  9:  Initial  values  for  b^  (Omit  if  IT  =  2  or  IB  =  0 

on  card  2) 

card  a  cols.  1-80  Format  for  reading  in  b's.  The  b's  must  be 

read  as  floating-point  numbers. 

card(s)  b  b(i)  ,  (  i  =  1, . ..,  LITTLN  )  -  Punched  in  the  format 

specified  in  card  a. 

CAKD(S)10:  Initial  values  for  c^  (Omit  if  IT  =  2  or  NCH  ^  10 

on  card  2) 

c(i),  (1*1,...,  LITTLN)  -  Punched  in  8FIO.7  format. 
Note:  If  the  ^  are  read  in,  then  the  limits  on  the  parameters 


must  be  read  in,  i.e.,  LIM  ^  0  on  CARD  2. 


CARD(S)  11: 


COLS 

FORMAT  (8F10;0) 

(Omit 

1-10 

AL0 

= 

lower 

limit 

on 

ai 

11-20 

AHI 

= 

upper 

limit 

on 

a. 

1 

21-30 

BL0 

- 

lower 

limit 

on 

b. 

X 

31-1+0 

BHI 

= 

upper 

limit 

on 

b. 

X 

1+1-50 

blank 

51-60 

CHI 

= 

upper 

limit 

on 

Ci 

61-70 

TL0 

= 

lower 

limit 

on 

9 

a 

71-80 

THI 

= 

upper 

limit 

on 

9 

a 
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Restart  Procedure 

The  program  may  be  restarted  using  the  punched  output  from  a  previous 
run.  It  is  possible  to  add  or  remove  examinees,  change  6's,  change  item 
parameter  estimates,  and  add  or  remove  items  from  the  output  after  it  has 
been  read  in. 

If  no  changes  are  to  be  made  in  the  output,  the  following  modifications 
must  be  made  to  the  setup  to  restart  after  the  first  run  on  a  set  of  data. 

a.  If  any  of  the  examinees  were  removed  from  the  previous  run 
because  they  had  perfect  scores,  decrease  N  on  the  first 
card  by  the  number  of  examinees  removed.  Modify  card  6  to 
add  these  examinees  to  the  examinees  being  removed  from  the 
input  file. 

b.  On  card  2  make  the  following  additions.  Set  col.  17-20 
to  2000,  col.  25  to  1,  col.  55  to  2,  and  col.  80  to  1. 

c.  Substitute  the  following  for  card  set  5*. 

1.  Modification  card,  with  col.  2  set  to  1. 

2.  The  punched  output  from  the  previous  run. 

d.  Card  11  must  be  included.  The  limits  punched  are 

the  limits  set  by  the  program  (see  description  of  card  2,  col. 

80  in  the  setup). 

Subsequent  restarts  only  require  that  the  punched  output  from  the 
previous  run  be  replaced  with  the  new  punched  output. 

The  following  pages  describe  how  to  add  or  remove  thetas  and  the  corre¬ 
sponding  examinees,  add  or  remove  items,  change  item  parameters  or  change 
thetas  when  reading  in  the  output  of  a  previous  run.  If  no  changes  are 
necessary,  skip  to  the  section  titled  "Item  Response  Input"  on  page  ei8. 
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The  cards  for  the  changes  follow  the  punched  output  from  a  previous 
run.  The  cards  must  be  in  the  order  of  their  description  here.  As  an  added 
reminder  the  cards  will  be  numbered  starting  with  set  c3  in  the  order  that 
they  should  be  placed.  If  a  specific  change  isn’t  made,  the  cards  for  that 
change  are  omitted. 

The  modification  card  mentioned  in  cl  on  page  B9  contains  an  indicator 
for  each  change  that  will  be  set  to  1  if  the  change  is  required,  blank  if 
no  change  is  required.  A  brief  description  of  the  card  follows. 

COLS  FORMAT  (4012) 


2 

INP 

= 

0 

0’s  are  punched  in  E  format 

= 

1 

0's  are  punched  in  hexadecimal  in  A4 

format 

b 

JADTH 

= 

1 

More  0's  are  to  be  estimated  than  those 

read 

6 

JSBTH 

= 

1 

Remove  0 ' s  from  the  0 ' s  read 

8 

JCHTH 

= 

1 

Change  some  0 ' s  read 

10 

JADIT 

= 

1 

Add  more  items  than  read 

12 

JSBIT 

= 

1 

Remove  some  of  the  items  from  the  input 

14 

JCHIT 

= 

1 

Change  the  a's,  b's  and  c's  for  some  items 

16 

JCHA 

= 

1 

Change  the  a's  for  some  of  the  items 

18 

JCHB 

= 

1 

Change  the  b's  for  some  of  the  items 

20 

JCHC 

= 

1 

Change  the  c's  for  some  of  the  items 
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The  following  changes  are  necessary  to  add  or  remove  examinees 
and  their  corresponding  thetas  or  to  change  thetas  for  some  examinees.  In 
order  to  describe  these  changes  a  numerical  example  is  essential.  Suppose 
that  in  the  previous  run  examinees  21,  65,  and  250  have  been  removed  from 
the  item  response  data  set.  These  numbers  are  the  positions  of  the 
examinees  on  the  input  data  set  of  item  responses.  Card  5  reads: 


cols. 

5 

5 

9-10 

21 

14-15 

65 

18-20 

250 

21-25 

blank 

Rest  of  card  as  described  in  setup. 

The  0’s  computed  and  printed  will  have  the  following  relationship  to  the 
examinees  on  the  data  set  of  item  responses. 


Examinee 

Position  in  ©' 

1 

1 

22 

21 

64 

63 

66 

64 

249 

247 

251 

248 

A.  To  change  9's 

1.  Set  JCHTK(col8  of  the  modification  card)  to  1. 

2.  Add  card  set  c3  punched  as  follows. 


COLS 

Card  a  1-5 

N$UT  -  Number  of  0's  to  be  changed 

Card  b 

4(15,  E15.8) 

1-5 

index  of  first  0  to  be  changed 

6-20 

new  value  for  that  0 

21-25 

index  of  2nd  0  to  be  changed 

26-40 

new  value  for  that  0 
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The  index  of  the  9's  corresponds  to  the  indices  listed  under  the 
position  in  6  column  above  and  corresponds  also  to  the  index  in 
the  listing  of  final  0's  printed  in  the  corresponding  output. 

Add  thetas  for  examinees  that  have  previously  been  removed. 

1.  N  on  card  1  is  the  previous  N  plus  the  number  of  thetas 
being  added. 

2.  Set  JADTH  (col.  4  of  modification  card)  to  1. 

3.  Add  card  set  c4  punched  as  follows. 


Card  a 

COLS 

1-5 

N$UT  -  Number  of  0's  to  be  added 

Card  b 

4(15,  E15.8) 

1-5 

Position  that  the  first  added  0  will  have 

6-20 

in  the  new  sequence 

Initial  value  for  the  first  added  0 

etc. 

etc.  (0's  must  be  punched  in  ascending 

order  by  position) 

Using  our  example  given  above,  suppose  we  'want  to  add  examinees  21 
and  250.  The  positions  that  these  examinees  will  have  in  the  new  list 
of  0's  is  21  and  249  respectively.  Thus  N$UT  equals  2,  card  c4  contains 
21  in  cols.  4-5,  cols.  6-20  contain  the  initial  value  of  theta  for  ex¬ 
aminee  21,  and  23-25  contains  249  and  cols.  26-40  contain  the  initial 
value  of  theta  for  examinee  250. 

4.  Card  6,  set  MILL  equal  to  MILL  minus  the  number  of  added  0's, 
and  remove  the  examinee  numbers  from  the  KILL  vector.  In  the 
example,  MILL  will  now  be  1,  and  KILL(l)  =  65  « 


Remove  0's  and  corresponding  examinees. 


1.  N  on  card  1  becomes  the  previous  value  of  N  minus  the  number  of 
examinees  to  be  removed. 

2.  Set  JSBTH  (col.  6  of  modification  card)  to  1. 

3.  Add  card  set  c5  punched  as  follows. 

COLS  (1615) 

1-5  N0UT  -  Number  of  9’ s  to  be  removed 

6-10  Sequence  number  of  first  9  to  be  removed 

(numbers  should  be  in  ascending  numerical  order) 

11-15  Sequence  number  of  second  9  to  be  removed 

etc .  '  etc . 

The  sequence  number  is  the  position  of  the  theta  in  the  input 
vector  of  thetas.  Referring  back  to  the  example  given  in  the 
beginning  of  this  section  suppose  that  we  want  to  remove  the  15th 
and  the  80th  theta.  Card  set  c5  would  then  contain 

col.  (1615) 

5  2 

9-10  15 

14-15  80 

k.  Card  6.  Since  we  are  removing  two  more  thetas  we  must  also  remove 
their  associated  examinees.  The  examinee  number  corresponding  to 
the  15th  9  is  15,  that  corresponding  to  the  80th  9  is  82. 

Card  5  should  read,  referring  back  to  the  initial  example. 

cols. 

5  5 

9-10  15 

14-15  21 

19-20  65 

24-2J  82 

28-30  250 

31-35  blank 

Rest  of  card  as  described 
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The  following  cards  describe  the  addition*  1"  the  *  p',vll’  to 
remove  Items  or  change  the  parameters  for  an  item.  'the  nvunbe*  o(  the  H.em  in 
it 9  item  number  rend  In  Input  card  I*. 

A.  Add  items. 

1.  Set  , TAD  IT  (col.  10  of  modification  card)  to  1. 

2.  Add  card  set  ch  punched  as  follows. 


COLS 

card  a 

1-5 

N0UT  -  number  of  item#  to  be  added 

card  b 

(15.  31)15.8,  If)) 

1-5 

Number  of  test  item  to  be  added 

0-20 

Initial  a  value  for  that  item 

21-35 

Initial  b  value  for  that  item 

36-50 

Initial  c  value  for  that  item 

51-55 

ITESTj  number  of  parameters  to  vary 

5.  The  next  run  that  uses  the  output  of  this  run  will  require  that, 
n  on  card  1  be  the  n  of  this  run  plus  the  number  of  items 

added  and  the  item  numbers  of  the  added  items  must  be  added 
to  card(s)  4. 

B.  Subtract  Items. 

1.  Set  JSBIT  (col.  12  of  modification  card)  to  1. 

2.  Add  card  set  c7  punched  as  follows. 

COLS  (1615) 

1-5  N0UT  -  Number  of  items  to  be  removed 

6-10  Number  of  1st  item  to  be  removed 

11-15  Number  of  2nd  item  to  be  removed 

etc.  etc. 

*The  number  of  parameters  allowed  to  vary  is  1  if  only  b,  is  allowed 
to  vary,  2  if  and  are  allowed  to  vary,  3  if  a.  ,  b,  ,  and  c. 

are  allowed  to  vary.  1 
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5.  Tit*  next.  run  that  usex  the  output  of  thin  run  will  require 
the  following  ohMitM«*»* i  (a)  it  oh  card  l  will  be  the  it  of 
thl*»  ntn  ml  mm  Urn  number  of  Items  removed  and  (It)  the  Item 


numbers  foi*  Um  items  removed  muni  it*  removed  from  the  Item 
numbers  oit  uard(a)  4,  (usually  requires  repunohlng). 

0.  Change  *U  parameters  for  « nmp  items  • 


l«  del,  ill' HIT  (ho, l ,  lit  of  mod  if  ioation  uard)  to  1. 
2.  Add  hill'd  aot  hll«  punch  a*  desuribed  below 


COW 

card  a  l'*> 

N^lIT  -  N»mb*r  of  item*  uhanged 

hard(a)  b 

(15»  5K15,a,  15) 

1-5 

Number  of  item  for  which  th*  item  parameters 

arc  to  be  changed 

6-20 

New  a 

21-55 

New  b 

56-50 

New  e 

51-55 

rr«aY(i)  }  number  of  parameters  allovmd  to  vary* 

D.  Chang*  only  some  a’ a. 

1,  8«t  JCHA  (eol.  16  of  modification  oard)  to  1. 

2.  Add  card  aet  o<?,  punched  aa  d*acrib«d  b*low. 


COLC 

card  a  1-5 

N^OT  -  Number  of  items  for  which  the  initial 

values  are  to  be  changed 

card(s)  b 

(4(15,  E15.8)) 

1-5 

Number  of  item  for  which  a  is  to  be  changed 

6-20 

Now  value  for  a 

is  allowed 
and  are 
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0QU1 

21-25 

Number  of 

next  item  for  which  • 

changed 

26-40 

New  valua 

for  that  a 

etc . 

etc . 

K.  Change  only  b'*, 

L.  Art  JOHN  (col.  18  of  modification  card)  to  1, 
2»  Add  card  aet  cUO,  punched  as  daacribed  below. 
C0I<3 


card  a  l-§ 
card (a)  b 


N^IP,P  -  Number  of  b'  a  to  be  changed 

Elii.8)) 


1-5  Number  of  let.  item  for  which  b  is  to  be 

changed 

6-20  New  value  of  b  for  that  item 

21-25  Number  of  2nA  item  for  which  b  is  to  be 

changed 

26-40  New  value  of  b  for  that  item 

etc.  etc. 

F.  Change  only  the  c's. 

1,  Set  JCHC  (col#  20  of  modification  card)  to  1. 

2,  Add  card  set  ell,  punched  as  described  below. 


COLS 

card  a  1-5 
card(s)  b 

1-5 

6-20 


N0UT  -  Number  of  c's  to  be  changed 
(4(15,  El.5.8)) 

Number  of  1st  item  for  which  c  is  to 
be  changed 

New  value  of  c  for  that  item 
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COIfl 

21-25  Number  of  2nd  item  for  which  initial  c 

is  to  be  changed 

26-40  New  value  of  c  for  that  item 

Q.  To  change  the  number  of  parameters  that  are  allowed  to  vary  for  an 
item,  either  change  the  punched  output  (the  number  of  parameters  that 
are  allowed  to  vary  are  punched  16  per  card  in  1615  format)  or  follow 
the  procedure  for  changing  all  of  the  parameters  for  an  item,  see  C 
on  page  B15,  punching  the  current  values  for  a  ,  b  ,  and  c  but 
changing  the  number  of  parameters  allowed  to  vary. 


-bi8- 


Item  Response  Input 

The  item  responses  for  a  test  can  be  on  disk  or  tape  in  BCD  or  binary 
with  one  record  per  examinee .  The  examinees  are  numbered  in  the  order  that 
they  are  read.  Specific  examinees  can  be  omitted.  Examinees  skipped  at  bhe 
beginning  of  the  data  set  or  when  selecting  every  k  examinee  are  not  included 
in  the  numbering.  The  program  reads  the  responses  from  unit  10.  Under  OS  360, 
the  responses  can  be  on  cards  and  read  on  unit  10.  However  for  debugging 
purposes  the  option  to  read  card  input  with  the  setup  is  included. 

Each  record  contains  the  item  responses  coded  1  if  right,  0  otherwise, 

til 

with  item  1  the  first  digit  in  the  record  and  the  last  item  is  the  m 
digit  in  the  record,  for  m  test  items.  The  items  to  be  used  are  specified 
on  card  4  of  the  setup.  For  example  if  there  are  10  responses  per 
examinee  and  only  the  2nd,  5th,  7th,  and  8th  are  to  be  used,  then  card  4 
would  contain  2  in  col.  3,  5  in  col.  6,  7  in  col.  9,  and  8  in  col.  12. 

The  input  options  in  card  2  are 

1.  INPUT  =  0.  The  item  responses  are  on  unit  10  in  BCD. 

2.  INPUT  -  3*  The  item  responses  are  on  unit  10  in  binary. 

3.  INPUT  =  2.  The  item  responses  are  punched  packed  ready  for  use 

one  card  per  examinee.  An  A4  format  for  reading  the  responses 

is  read  in  at  execution  time.  No  examinees  may  be  skipped  or  items 


selected. 


Printed  Output 


1st  Page 

1.  Number  right  score  -  mean  and  variance  followed  by  the  number  right  scores 
for  each  examinee  printed  20  per  line,  after  the  first  number  in  the  line 
which  is  the  sequence  number  for  the  first  examinee  in  that  row. 

2.  If  input  is  output  from  a  previous  run,  the  title,  no.  of  examinees,  no. 
of  items,  time  and  date  of  the  previous  run  is  printed. 

3.  If  any  examinees  are  skipped,  their  sequence  numbers  are  printed. 

4.  .  If  the  input  is  binary,  the  total  input  record  is  printed  in  hexadecimal 

for  the  first  10  examinees,  followed  by  the  packed  responses  for  the 
items  selected  printed  in  hexadecimal.  The  words  in  which  the  selected 
responses  are  stored  are  not  zeroed  beforehand.  Consequently  if  the 
number  of  item  responses  does  not  fill  a  whole  word,  the  remainder  of 
the  word  will  be  garbage. 


2nd  Page 
1st  line: 
2nd  line: 


3rd  line: 


4th  line: 


5th  line: 


6th  line: 


title,  number  of  items,  number  of  examinees,  time  and  date. 

maximum  number  of  stages,  maximum  number  of  iterations  for  items, 

and  NCH,  the  number  of  choices.  If  the  c's  are  read  in,  the 

number  of  choices  will  print  out  asterisks. 

the  number  of  items  that  may  be  removed  if  their  a ' s  become 

greater  than  the  limit  for  a  that  has  been  read. 

number  of  people  removed  because  they  had  perfect  scores,  followed 

by  their  sequence  numbers. 

if  the  initial  item  parameters  are  computed,  a  list  of  the 
percentage  of  people  who  got  the  item  right  is  printed, 
initial  estimates  for  a  ,  b  ,  and  c  ,  and  the  number  of  item 


parameters  that  are  allowed  to  vary#  2  means  a  and  b  are 
allowed  to  vary,  3  that  a  ,  b  ,  and  c  vary,  1  that  only  b 
varies. 

7th  line:  initial  estimates  for  Q's,  printed  12  per  line,  the  first  number 
in  a  line  is  the  sequence  number  of  the  first  theta  in  the  line. 
Note:  The  sequence  numbers  are  from  1  to  the  number  of  examinees 
minus  the  number  of  the  examinees  removed. 

8th  line:  initial  likelihood  function  is  printed. 

Page  2 

1.  "Beginning  of  stage"  followed  by  stage  number. 

2.  Thetas  with  absolute  value  greater  than  3*  If  there  are  none,  only 

the  message  will  be  printed. 

3.  Normalized  a^  and  ,  i  =  1,2, ...,n  . 

4.  Information  about  the  normalization  of  9's. 

P(2/3)  is  the  proportion  of  9's  minus  X(2/3)  with  absolute  values 
less  than  2/3. 

P(115)  is  the  proportion  of  9's  minus  X(2/3)  with  absolute  value 
less  than  1.15. 

X  is  the  scaling  factor  -  refer  to  the  description  of  the 

program  for  more  information. 

P(-2/3)  is  the  proportion  of  9's  such  that  -2/3  <  9  <  0  . 

P(2/3)  is  the  proportion  of  9's  such  that  0  <  9  <  2/3  * 

X(2/3)  is  the  mean  of  9's  for  9^  ,  such  that  -2/3  <  9^  <  2/3  . 

5*  Time  for  computation  of  9's  is  printed. 


printout  isn't  requested,  a  line  described  by  the  title  at  the  top  of 
page  3  is  printed.  This  contains 
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a.  Item  number 

b.  Derivatives  of  the  log  of  the  likelihood  function  for  a  ,  b  ,  and  c 

c.  Deltas  for  a  ,  b  ,  and  c 

d.  Iteration  number 

e.  An  if  the  maximum  number  of  iterations  has  been  reached 

f.  If  a  is  fixed,  the  value  that  a  is  fixed  at  is  printed 

g.  If  c  is  fixed,  the  value  that  c  is  fixed  at  is  printed  and 

whether  it  was  fixed  by  exceeding  the  upper  or  lower  bound. 

3*  Following  all  the  item  iterations,  the  new  values  of  a  ,  b  ,  and  c 
are  printed. 

Page  4 

1.  Maximum  derivatives  for  a  ,  b  ,  and  c  and  the  maximum  number  of 
iterations  is  printed. 

2.  Maximum  theta  derivative  and  maximum  number  of  iterations  for  the  thetas. 

3.  The  likelihood  function  after  the  last  Q  estimations  is  printed. 

4.  Time  for  the  total  item  iterations. 

5*  "End  of  stage"  followed  by  the  stage  number. 

6.  Total  time  for  the  stage. 

Page  3 

1.  If  any  ty.  >  19  ,  and  the  corresponding  x,  is  1,  a  message  is 

lcl  let 

printed  giving  the  index  of  the  ability.  Ignore  this  message. 

2.  Likelihood  function  after  change  in  item  parameters. 

3.  Message  that  the  likelihood  function  has  increased,  if  it  has  since  the 
initial  computation  of  the  likelihood  function  or  since  the  previous 
computation  of  the  likelihood  function  after  a  change  in  the  item 
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parameters.  If  the  likelihood  function  hasn't  increased,  no  message 
is  printed. 

Repeat  from  page  2  through  page  5  until  either  the  procedure  con¬ 
verges,  the  maximum  number  of  stages  has  been  reached,  or  the  time 
limit  is  exceeded. 

Convergence  is  determined  -when  the  likelihood  function  computed 
after  a  change  in  item  parameters,  has  decreased  in  two  out  of  four 
stages.  After  it  has  converged,  two  more  stages  are  done  with  detailed 
printout.  If  the  maximum  number  of  stages  has  been  reached  or  there 
isn't  enough  time  for  another  stage,  the  program  ends  after  the  item 
parameter  estimation  when  it  prints  "Time  for  Stage." 
last  Page 

At  the  end  of  the  run  the  final  9’s  are  printed,  12  9' s  per  line, 
with  the  first  number  in  the  line  the  sequence  number  of  the  first  9 


in  the  line 


Punched  Output 


At  the  end  of  a  run  the  following  cards  are  punched  so  that  the  program 
can  be  restarted. 

Card  1 

1-60  Title 

64-65  Number  of  items 

66-72  Number  of  examinees 

Card  2 

Date  and  time  of  run  (5A4) 

Card  Set  3  (l4,  19A4) 

O's  punched  19  per  card.  The  first  4  columns  contain  the  sequence 
number  for  the  card,  the  0's  are  punched  in  19A4  format. 

Card  Set  4 

Format  for  the  a's  which  is  (5E15*8). 

The  a's,  punched  in  5EI5.8  format. 

Card  Set  5 

Format  for  the  b's  (5EI5.8). 

The  b's  punched  in  5E15«8  format. 

Card  Set  6 

The  c's  punched  in  8F10.7  format. 

Card  Set  7 

The  number  of  parameters  that  are  allowed  to  vary  for  each  item 
punched  in  1615  format. 


